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ABSTRACT 



The National Institute on Alcohol Abuse and Alcoholism 



established the Task Force of the National Advisory Council on Alchohol Abuse 
and Alcholism to review and report on the existing research on college 
student drinking, including the evaluation of camps and community policies, 
prevention programs, and early intervention strategies. This review has been 
published as A Call to Action: Changing the Culture of Drinking at U.S. 
Colleges. This brief guide is intended to provide some direction as to how. 
this research can be incorporated most effectively into an explicit planning 
process to maximize the impact of any prevention strategy and to monitor any 
interventions implementation and local impact. The guide is organized around 
the five basic steps of effective prevention planning and evaluation: (1) 

identifying specific goals and objectives; (2) reviewing the evaluation 
research; (3) outlining how the intervention will work; (4) creating and 
executing a data collection plan; and (5) providing feedback to the 
intervention program. The guide concludes with references that relate to 
topics of alcohol-related problems among college students. (Contains 29 
references.) (SLD) 
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Executive Summary 



The National Institute on Alcohol Abuse and 
Alcoholism (NIAAA) is committed to helping colleges 
and universities reduce alcohol-related problems on their 
campuses, protect students from harm, and improve 
quality of life for the entire campus community. To 
guide future efforts, the Advisory Council on Alcohol 
Abuse and Alcoholism established a Task Force on 
College Drinking to review and report on the existing 
research on college student drinking, including 
evaluations of campus and community policies, 
prevention programs, and early intervention strategies. A 
summary of the Task Force s report, A Call to Action: 
Changing the Culture of Drinking at U.S. Colleges, 
provides college administrators and program specialists 
with a useful overview of these evaluations, which can be 
used to inform future program and policy development 
(see http://www.collegedrinkingprevention.gov for 
complete text of the report). 

This brief guide is intended to provide some direction 
as to how this research can be incorporated most 
effectively into an explicit planning process to not only 
maximize the impact of any prevention strategy, but also 
to actively monitor any interventions implementation 
and local impact. By tightly integrating evaluation into 
prevention planning and management, college 
administrators and program staff can assure themselves 
that objectives are clear to all, and that precious 
resources are being spent effectively. More broadly, our 
hope is that, when greater numbers of college and 
university administrators commit their institutions to 
sound planning and evaluation, all of us will benefit 
from their work. The following paragraphs show how 
the guide is organized. 



Steps for Effective 
Prevention Planning 
and Evaluation 

Thinking about the evaluation as part of the planning 
process will sharpen everyone’s thinking about the 
program: its mission, its goals, its objectives, and the 
activities designed to meet those objectives. The process 
for developing and evaluating prevention programs and 
policies can be divided into five basic steps: 

1. Identifying specific goals and objectives 

2. Reviewing the evaluation research 

3. Outlining how the intervention will work 

4. Creating and executing a data collection plan 

5. Providing feedback to the intervention program. 

1. IDENTIFY SPECIFIC GOALS 
AND OBJECTIVES 

The problem — student drinking — is obvious, but exactly 
which goals and objectives should be specified to guide 
campus prevention efforts is not. Is the goal to eliminate 
college student drinking? Limit excessive consumption of 
alcohol? Eliminate alcohol-related behavior problems? 
Protect student drinkers from harm? Should the 
prevention effort focus on student drinking on campus, 
or should it also cover off-campus behavior? How college 
officials answer these questions will depend on several 
factors: the philosophy and academic mission of the 
institution, the nature of the student alcohol problem, 
the level of prevention resources available, the views and 
opinions of key constituencies, the characteristics of the 
surrounding community, and the cultural and political 
context in which the school operates. 
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Having an evaluator be part of the planning process 
from the beginning will help guarantee that staff have 
listed out specific goals and objectives. In turn, these 
goals and objectives can be translated into specific 
outcomes that are assessed through the evaluation 
process. The evaluator can help a college s officials reach 
consensus on their specific goals and objectives. This is a 
good example of how planning an intervention can help 
shape the intervention as well. 

2. REVIEW RESEARCH ON COLLEGE 

DRINKING INTERVENTIONS 

The next step is to review program and policy options 
that might be applied to achieve the outlined goals and 
objectives. We present a typology of prevention 
interventions that comprises programs and policies 
classified into one of the following levels: 1) individual, 

2) group, 3) institution, 4) community, and 5) State and 
Federal public policy. Many areas of strategic 
intervention can be pursued at one or several levels of 
the social ecological framework. Implementing multiple 
strategies from these various levels would greatly increase 
the likelihood of the objective being achieved. 

In this section we summarize some of the major 
findings from a review of the literature on college- 
focused prevention, organized according to the levels of 
intervention (1-5 above). 

3. OUTLINE HOW THE 

INTERVENTION WILL WORK 

A review of available research, plus consultations with 
other college and university prevention specialists, will 
suggest a set of program and policy options that can be 



adopted. The next planning step is to outline the chain 
of events that will lead from implementation of each 
component program or policy to its specific (and 
measurable) objective. This is often called building the 
“logic model” for the intervention. We provide a simple 
example of the kind of flow chart that is often the 
clearest and most economical way of presenting this 
information (see Figure 1). 

There are several reasons why this step is important: 

• First, developing the logic model will pinpoint areas 
of uncertainty, confusion, or disagreement among 
members of the planning team. 

• Second, work on the logic model can make 
transparent any false assumptions that need to be 
addressed. 

• Third, development of the logic model will help 
guarantee that all program activities and policies can 
be logically linked to the achievement of specific 
objectives. 

• Fourth, a logic model can later serve as an 
educational and communications tool when a new 
program or policy is being implemented. 

• Fifth, a logic model can be a tool for tracking 
changes in the intervention or its implementation. 

• Finally, the logic model helps inform the evaluation 
so that it can answer the fundamental question of 
whether the program effects were smaller (say) 
because the fundamental concept behind the 
intervention was wrong, the implementation was 
flawed, or one piece of the intervention sequence 
fell apart. 
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4. CREATE AND EXECUTE 

A DATA COLLECTION PLAN 

Self-report surveys are a primary data source for program 
and policy evaluations, especially if the goal is to reduce 
consumption or alcohol-related problem behaviors. If a 
student survey is to be part of an evaluation, we briefly 
describe some basic requirements of a valid and useful 
survey. It should be remembered, however, that a 
student survey is not the only source of useful data, and 
in some cases may not even be the best source. Ideally, 
colleges and universities will put in place a system for 
recording a wide range of alcohol-related incidents 
involving students. These might include data from 
urgent or emergency care facilities, campus police 
student counseling services, residence halls, athletic 
departments, and offices of student discipline. 

On many campuses, the problem is that data are 
recorded but are not easily accessed, but this situation is 
improving as offices move toward using computerized 
databases and automated data entry. As these systems are 
put in place, administrators should be sure that records 
of campus problems make note of alcohol involvement. 

5. COMMUNICATING EVALUATION 

RESULTS: FEEDBACK 

The full value of any evaluation is only realized when it 
can provide ongoing feedback to the program and the 
affected community at large. Often, this feedback is 
necessary just to create support for the program or 
intervention to be continued. Important information on 
individual program components may also prove valuable 
for continuously improving the intervention or for 
guarding against degradation in the programs impact. 



Program Evaluation: 

The Big Picture 

Newcomers to the topic of college student drinking are 
often puzzled to learn that the field’s knowledge of “what 
works” is relatively slim. Apart from some recent and 
promising interventions aimed at individual students, 
the conscientious program planner will find little 
empirical evidence to guide choices of program and 
policy interventions aimed at the broader college 
population. The broader field of prevention research, 
which has examined the impact of programs and policies 
aimed at youth in the general population, provides 
useful guidance. Even so, it is clear that evaluations of 
environmentally focused prevention strategies that focus 
specifically on college populations are sorely needed. 

We are urging higher education administrators to 
incorporate evaluation as an integral part of program 
planning, which we view to be essential to developing 
more effective prevention programs and policies. We 
hope that administrators will realize that the evaluations 
they undertake will also contribute significantly to our 
knowledge of “what works.” Conducting and then 
sharing the results of evaluations of alcohol prevention 
efforts is necessary to meet that larger goal. 

Resources 

The guide concludes with a number of references, both 
publications and Web sites, that directly relate to the 
topic of alcohol-related problems among college 
students. 
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Introduction 



The National Institute on Alcohol Abuse and Alcoholism 
(NIAAA) is committed to helping colleges and 
universities reduce alcohol-related problems on their 
campuses, protecting students from harm, and improving 
quality of life for the entire campus community. To guide 
future efforts, the National Advisory Council on Alcohol 
Abuse and Alcoholism established a Task Force on 
College Drinking to review and report on the existing 
research on college student drinking, including 
evaluations of campus and community policies, 
prevention programs, and case referral and intervention 
systems. The Task Force s Report, A Call to Action: 
Changing the Culture of Drinking at US. Colleges, 
provides college administrators and program specialists 
with a useful overview of this review and evaluation of 
available research findings, and can be used to inform 
future program and policy development (see 
http://www.collegedrinkingprevention.gov for complete 
text of the report) . 

While informative, this evaluation of the research 
literature cannot be translated into a simple formula for 
college and university administrators and program staff 
to follow. School officials can learn from what others 
have tried, but ultimately they must devise, through 
successive approximation, a tailored approach that fits 
the needs of their own institution. 

This means, therefore, that each institution must have 
a feedback mechanism in place for monitoring its 
prevention programs and policies and for evaluating 
their effectiveness. A well-managed corporation monitors 
its business operations and sales. Likewise, colleges and 
universities should ensure that their prevention programs 
and policies are being implemented as planned and are 
working well to reduce alcohol-related problems. 
Otherwise, substantial financial and staffing resources 



will continue to be expended without knowledge of 
whether the programs implemented are actually 
ameliorating the problem. 

Our objectives in writing this guide are simple. First, 
we want college and university administrators and program 
specialists to know that there is evaluation research 
available that can guide the overall direction of prevention 
work on their campus. We believe it is essential that top 
administrators insist their staffs consult this research 
literature when designing new programs and formulating 
new policies, rather than rely on conventional wisdom 
or tradition to guide them. Simply replicating what 
other schools are doing is not a substitute for sound 
planning. 

Second, we want administrators and program staff to 
understand better the central role of evaluation in 
planning. As we outline below, the ideal is for program 
planning and evaluation to be tightly integrated. Top 
college and university administrators, and the governing 
boards or State legislatures to whom they report, are in a 
strong position to urge adoption of this integrated 
approach. They can insist that prevention planning be 
guided by clearly articulated goals, objectives, and 
activities, all informed by current research. They can 
provide the resources needed for data collection and 
analysis. And they can foster a supportive atmosphere 
where evaluation is used as a learning tool, not as a 
weapon for threatening elimination of programs or staff 
positions. 

More broadly, our hope is that, when greater numbers 
of college and university administrators and program 
staff commit their institutions to sound planning and 
evaluation, all of us will benefit from their work. 



er|c 



7 



5 



Steps for Effective Prevention 
Planning and Evaluation 



Unfortunately, because most administrators associate 
evaluation with the measurement of final results, 
prevention planners often fail to think about evaluation 
until after their programs and policies are up and 
running. Instead, the evaluation should be planned as 
the prevention program is being developed. 

Thinking about evaluation as part of the planning 
process will sharpen everyone's thinking about the 
program: its mission, its goals, its objectives, and the 
activities designed to meet those objectives. Used in this 
way, evaluation planning can be a valuable management 
tool. Many prevention planners are finding it useful to 
view program and policy development and evaluation as 
an iterative process, with evaluation findings informing 
later alterations. 

The process for developing and evaluating prevention 
programs and policies can be divided into five basic steps: 

1. Identifying specific goals and objective 

2. Reviewing the evaluation research 

3. Outlining how the intervention will work 

4. Creating and executing a data collection plan 

5. Providing feedback to the intervention program. 

Basic considerations for each step are described below. 
We realize that program planning does not always 
proceed with these steps in sequence. Indeed, it is typical 
for earlier steps to be revisited as planners refine their 
thinking. 

1. IDENTIFY SPECIFIC GOALS AND 
OBJECTIVES 

The problem — student drinking — is obvious, but exactly 
which goals and objectives should be specified to guide 



campus prevention efforts is not. Is the goal to eliminate 
college student drinking? Limit excessive consumption of 
alcohol? Eliminate alcohol-related behavior problems? 
Protect student drinkers from harm? Should the 
prevention effort focus on student drinking on campus, 
or should it also cover off-campus behavior? 

How college officials answer these questions will 
depend on several factors: the philosophy and academic 
mission of the institution, the nature of the student 
alcohol problem, the level of prevention resources 
available, the views and opinions of key constituencies, 
the characteristics of the surrounding community, and 
the cultural and political context in which the school 
operates. 

This is the time at which one would conduct a so- 
called “needs assessment.” While this assessment may be 
conducted with varying degrees of formality, and 
comprise focus groups, large-scale surveys, or open- 
ended interviews with members of the (extended) 
campus community, the purpose is to better understand 
not only the nature and scope of the problem for any 
specific campus, but ideally, to also understand the 
broader social, economic, and physical context that 
might shape both the problem and the range of 
appropriate strategies that might be adopted to reduce it. 
Conducting a needs assessment and communicating its 
results may also be required in order to achieve agreement 
among key campus stakeholders, including the students. 

Unfortunately, it is common for college officials to 
jump into prevention work without taking time to 
explore what their goals and objectives are. This often 
happens when program planners choose to replicate 
programs and policies from other campuses without 
thinking through exactly what they are intended to 
accomplish. 
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For example, consider social norms marketing 
campaigns. Using local campus media, these campaigns 
are designed to reduce heavy alcohol consumption by 
communicating accurate information about student 
drinking levels, thereby correcting the common 
misperception that “everyone” drinks. Such a program is 
less likely to be successful on a campus with older 
commuter students who do not identify with the 
institution or their student peers. In addition, such 
campaigns are also more likely to foster moderation 
rather than total abstinence, an outcome that may be 
philosophically unacceptable at some institutions. Even 
so, many prevention specialists have been swept along by 
the current enthusiasm for social norms marketing 
without considering whether such a program is a good 
Fit for their campus. 

There is the risk, then, that program planners may end 
up pursuing a set of disconnected programs and policies 
in service of multiple goals and objectives, some of 
which may even contradict one another. Instead, having 
an evaluator be part of the planning process from the 
beginning will help college officials consider a full range 
of options for what they want to accomplish, and have 
articulated specific goals and objectives. In turn, these 
goals and objectives can be translated into specific 
outcomes to be assessed through the evaluation process. 
This is a good example of how planning an intervention 
can also help shape the evaluation. 

Importantly, having an evaluator be part of the 
planning process will help a college s officials reach 
consensus on their specific goals and objectives. Absent 
this discussion, it is common for program planners to 
develop a broadly stated goal that can mean different 
things to different people. For example, a typical goal 



might be “to reduce student alcohol problems." Some 
officials will interpret that to mean abstinence is the 
goal, whereas other officials will think it means the 
development of programs and policies to protect 
students from the consequences of their heavy drinking. 
When evaluators are brought in to assess an established 
program, it is common for them to discover that 
different officials have widely varying opinions about 
what they are trying to achieve, a fact that had 
previously gone unrecognized. 

In listing goals and objectives, specificity is key. 
Evaluators will push program planners to develop precise 
and measurable objectives, meaning that the 
achievement of those objectives can be measured and 
readily observed. For example, stating that “alcohol 
consumption” will be reduced is too imprecise. Instead, 
a specific goal might be to increase the percentage of 
underage students who abstain from alcohol, or to 
decrease the number of separate occasions on which 
students consume alcohol per month, or to decrease the 
percentage of students who report having three or more 
drinks the last time they consumed alcohol. In some 
cases, it will be important to specify the time and place 
where the changes will be observed. For example, it 
might make sense to concentrate on reducing alcohol 
problems that occur during specific campus events (e.g., 
homecoming) or at off-campus taverns and bars. 

2. REVIEW THE RESEARCH ON 
COLLEGE DRINKING 
INTERVENTIONS 

The next step is to review program and policy options 
that might be applied to achieve the outlined goals and 
objectives. Many types of prevention programs and 
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The Social Ecological Framework 

Throughout this guide, we have used what some 
refer to as the Social Ecological Framework to 
provide an organizing scheme for campus 
interventions that run the gamut from individually 
targeted programs to campus, community, and even 
State-level prevention strategies. We find it a very 
useful rubric for an overview of this sort. 

This by no means rules out alternative 
perspectives, nor does it mean that one must adopt 
a fully comprehensive program that covers the 
entire range of interventions. Our advice, in fact, is 
to realistically assess what can be done with the 
resources available, and thus to maximize the 
chances of success in addressing a well-defined 
target. 

Having said that, however, we do feel that 
another great advantage of the social ecological 
framework is that it keeps one mindful of the 
broader context within which any specific 
intervention might be implemented. For 
example, how well could any conventional 
awareness campaign be expected to succeed on a 
campus where large and frequent parties start on 
Wednesday and run through Sunday? 



policies are being implemented in the name of reducing 
alcohol-related problems on campus. A typology of 
programs and policies developed by the U.S. 

Department of Educations Higher Education Center for 
Alcohol and Other Drug Prevention provides a useful 
way to categorize existing efforts, identify missing 
program elements, and guide new strategic planning. 

A social ecological framework used in public health 
work defines one dimension of the typology, with 
programs and policies classified into one of the following 



levels: 1) individual, 2) group, 3) institution, 

4) community, and 5) State and Federal public policy. 

The typology's second dimension concerns the key 
areas of strategic intervention, each of which is linked to 
a particular definition of the college alcohol problem. 
There are four alternative areas of strategic intervention 
to be considered: 

1. Changing peoples knowledge, attitudes, and 
behavioral intentions regarding alcohol 
consumption 

2. Eliminating or modifying environmental factors 
that contribute to the problem 

3. Protecting students from the short-term 
consequences of alcohol consumption (“health 
protection” or “harm reduction” strategies) 

4. Intervening with and treating students who are 
addicted to alcohol or otherwise show evidence of 
problem drinking. 

This typology is consistent with the “3-in-l 
Framework” to comprehensive and integrated prevention 
programs espoused by the NIAAA Task Force on College 
Drinking in its report, A Call to Action : Changing the 
Culture of Drinking at U.S. Colleges. The Task Force 
divided programs and policies according to three broad 
levels: 1) the student population as a whole; 2) the 
broader college and community environment; and 3) 
individual students. The value of both the “3-in-l ” and 
ecological frameworks is that they can be a useful 
introduction to encourage presidents, administrators, 
college prevention specialists, and community members 
to think in a broad and comprehensive fashion about 
college drinking. 
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Many areas of strategic intervention can be pursued at 
one or several levels in the social ecological framework. 
For example, consider intervention activities focused on 
the objective of increased observance and enforcement of 
the minimum drinking age law (also known as the age- 
21 law): 

• At the State or community level, the alcohol control 
commission could increase the number of decoy (or 
“sting”) operations at local bars and restaurants. 

• At the community level, local police could 
implement a protocol for notifying college officials 
of all alcohol-related violations involving students. 

• At the college itself, the campus pub could require 
that all alcohol servers complete a training course in 
responsible beverage service. 

• At the group level, the college might require that 
residential groups and special event planners provide 
adequate controls to prevent alcohol service to 
underage students. 

• At the individual level, a media campaign could 
publicize these new policies, the stepped-up 
enforcement efforts, and the consequences of 
violating the law. 

Implementing multiple strategies at these various levels 
would greatly increase the likelihood of the objective 
being achieved. 

Major findings from the review of the literature on 
college-focused preventions are presented below, 
organized according to the social ecological framework. 

Individual-Level Interventions. One set of programs 
is designed to increase student awareness of alcohol- 
related problems, change attitudes and beliefs, and foster 



each student s determination to avoid high-risk drinking. 
Typical among these efforts are freshman orientation, 
alcohol awareness weeks and other special events, and 
curriculum infusion, where faculty introduce alcohol- 
related facts and issues into their regular academic 
courses. The assumption behind these approaches is 
that, once students are presented with the facts about 
alcohols dangers, they will make better informed and 
therefore healthier decisions about drinking. Rigorous 
evaluations of these educational programs are rare, but 
work in elementary and secondary school-based settings 
suggests that, while these types of awareness programs 
are necessary, information alone is usually insufficient to 
produce behavior change. 

A second set of programs is designed to intervene with 
students whose pattern of alcohol use puts them at risk 
for serious negative consequences. There is little evidence 
that standard awareness and values clarification programs 
alone can reduce alcohol consumption by college 
students. There are new approaches being studied that 
do hold promise, however, including: 

1. Expectancy-challenge procedures. In this approach, 
didactic or experiential manipulations are used to show 
students that many of the effects they anticipate from 
drinking, such as sociability and sexual attractiveness, are 
due to their expectations, not to the alcohol per se. 
Students may be given a placebo drink but led to think 
that it contains alcohol, or they may observe others who 
have consumed alcohol or placebo drinks in a social 
setting. 

2. Cognitive-behavioral skills training. In these 
programs, students are taught several ways to reduce 
their risk of heavy drinking, including managing stress, 
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documenting daily alcohol consumption and planning 
ahead of time how much to drink before attending social 
events. Such programming works best when coupled 
with a motivational enhancement intervention. 

3. Brief motivational enhancement intervention. 
Delivered in a student health center, hospital emergency 
room, or other setting, this personalized intervention 
involves giving individual students feedback about their 
drinking behavior in comparison with others, 
information on the true drinking norms on campus, and 
a review of the negative consequences they are likely to 
suffer if they continue to drink at current levels. 

Research shows that receiving this feedback via computer 
rather than in-person can also work. 

As identified in the Task Force s report, these strategies 
have been shown to be effective, and will become more 
refined with further study to determine the most 
effective combination of program components. The 
ultimate challenge, however, may be in figuring out how 
to scale up these programs to impact the behavior of 
large numbers of students, not just a small number of 
research participants. 

Group-Level Interventions. Programs in this category 
have focused primarily on fostering peer-to-peer 
communication to change student social norms about 
alcohol use. Peer education programs, for example, train 
student leaders to implement a variety of awareness and 
educational programs and to serve as role models for 
other students. Well-structured evaluations of peer 
education are rare, so such programs remain an 
unproven strategy for reducing student alcohol 
consumption. 



Social norms campaigns are another prevention 
strategy in this category. This approach is grounded in 
the well-established observation that college students 
greatly overestimate the percentage of their peers who 
drink heavily. Because this misperception drives 
normative expectations about alcohol use, which in turn 
influence actual use, a viable prevention strategy is to 
correct the misperception. A social norms campaign 
attempts to do this by using campus-based mass media 
to provide more accurate information about actual levels 
of alcohol use on campus. Preliminary studies at several 
institutions suggest that this approach to changing the 
social environment may be promising, but more 
definitive research is still needed to gauge its impact. 

Most recently, there are a number of programs aimed 
at groups of students who share an affiliation (e.g., 
members of an athletic team, or fraternity members). 
Underlying such programs is the idea that prevention 
might take advantage of the social bonds formed by 
these groups to create peer sentiment for safer drinking 
practices. This is another area in which programs have 
outpaced evaluation, so little is known about how well 
these programs work. 

Institutional-Level Interventions. On campus, a task 
force should conduct a broad-based examination of the 
college environment, looking not only at alcohol-related 
policies and programs, but also at the academic program, 
the academic calendar, and the entire college 
infrastructure. The objective is to identify ways in which 
the environment can be changed to clarify the college's 
expectations for its students, better integrate students into 
the intellectual life of the college, change student norms 
away from alcohol and other drug misuse, or make it 
easier to identify students in trouble with substance use. 
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There are five strategic objectives that can be pursued 
at the institutional level: 

1. Offer and promote social, recreational, 
extracurricular, and public service options that do 
not include alcohol and other drugs 

2. Create a social, academic, and residential 
environment that supports health-promoting 
norms 

3. Limit alcohol availability 

4. Restrict marketing and promotion of alcoholic 
beverages 

5. Develop and enforce campus policies. 

Each of these objectives can be met through a variety 
of programs and policies. Consider limiting alcohol 
availability. Potential strategies include, among many 
others, banning or restricting the use of alcohol, banning 
delivery or use of kegs or other common containers, 
requiring use of registered and trained alcohol servers, 
and instituting responsible server training programs. 

Community-Level Interventions. Student alcohol 
problems are not a problem of the campus alone, but 
also of the surrounding community. Work at the 
community level can be accomplished through a campus 
and community coalition. Community mobilization, 
involving a coalition of civic, religious, and government 
officials, is widely recognized as key to the successful 
prevention of alcohol problems. Higher education 
officials, especially college and university presidents, can 
take the lead in forming these coalitions and moving 
them toward an environmental approach to prevention. 

A chief focus of a campus-community coalition should 
be to curtail youth access to alcohol and to eliminate 
irresponsible alcohol sales and marketing practices by 



local bars, restaurants, and liquor outlets. Potential 
strategies include limiting the number and concentration 
of alcohol outlets near campus, limiting the days or 
hours of alcohol sales, and instituting responsible server 
training programs. 

State and Federal Public Policy College officials 
should also work for policy change at both the State and 
Federal level. There are several potentially helpful laws 
and regulations that can be considered, including: 

1. Distinctive and tamper-proof licenses for drivers 
under age 21 

2. Increased penalties for illegal service to minors 

3. Prohibition of “happy hours” and other reduced- 
price alcohol promotions 

4. Restricted hours of sales 

5. Reduced density of retail outlets 

6. Increased excise tax rates on alcohol. 

Some communities have the ability to pursue these 
strategies locally through either local licensing laws, 
business permits, or through voluntary initiatives such as 
a code of responsible business practices that local owners 
and managers would agree to sign. 

3. OUTLINE HOW 

THE INTERVENTION WILL WORK 

A review of available research, plus consultations with 
other college and university prevention specialists, will 
suggest a set of program and policy options that can be 
adopted. The next planning step is to outline the chain 
of events that will lead from implementation of each 
component program or policy to its specific (and 
measurable) objective. Describing this chain of events is 
often called building the “logic model” for the 
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intervention. Very often drawing a flow chart is the 
clearest and most economical way of presenting this 
information (see Figure 1 for an example). 

There are several reasons why this step is important: 

First, a logic model will pinpoint areas of 
uncertainty, confusion, or disagreement among 
members of the planning team . It is common for 
different people to have varying theories about why a 
particular type of intervention should reduce student 
alcohol problems. In some cases, members of the 
planning team may even have a theory about why an 
intervention will have unintended negative effects. These 
competing expectations and theories need to be 
discussed and sorted out. 

Second, a logic model can expose any false 
assumptions that need to be addressed. For example, 
orientation programs for first-year students often assume 
that these students are unaware of alcohols dangers. 
Given that todays students have been given this basic 
information since they were in junior high school, that is 
very unlikely to be the case. However, there may be 
other legitimate reasons for providing this information. 
For example, students may not be aware of how heavy 
alcohol consumption can interfere with learning. 

Third, a logic model will help guarantee that all 
program activities and policies can be logically linked 
to the achievement of specific objectives. Items that 
cannot be so linked will be discarded from the plan. 

Work at this step may lead to a reconsideration of the 
objectives. For example, it may be that only a small 
number of intervention components can be organized to 
achieve a certain objective, due to considerations of 



available resources, political obstacles, or other barriers. In 
such cases, it might make sense to abandon that objective 
altogether and concentrate instead on other objectives 
that can be more easily achieved. 

Fourth, a logic model can later serve as an 
educational and communications tool when a new 
program or policy is being implemented. The logic 
model not only specifies what is being done, but 
also why. 

Fifth, a logic model can be a tool for tracking 
changes in the intervention or its implementation. As 
a concrete manifestation of institutional memory, a 
sequence of modified logic models will inform all parties 
to the intervention (including those new to the 
program) about how it was shaped along the way. 
Sometimes, a prevention team or administrator may 
need to be reminded of how the program or 
intervention was modified and for what reasons. 

A final reason for developing a logic model is that it 
helps inform the evaluation. In essence, the logic 
model makes clear the intervening steps that are 
hypothesized to lead from specific activities to specific 
outcomes. Data can be collected to document progress 
at each step. With this information in hand, evaluators 
can diagnosis what went wrong if a program or policy 
fails to meet its ultimate objective. 

Consider again the example of a social norms 
marketing campaign. The underlying premise of this 
campaign is that accurate information about drinking 
norms, conveyed through credible sources, will help 
students realize that the majority of their peers drink far 
less than they once thought was the case, which in turn 
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• Lower Prevalence of 
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Consequences 
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will reduce perceived pressure to drink heavily and drive 
down actual consumption. Figure 1 shows a simple logic 
model for this program. 

Starting from the left, the diagram shows a starting 
point at which students overestimate the prevalence of 
heavy drinking among their peers, a very common 
finding of student surveys. The educational campaign 
would be designed to change those beliefs, perhaps with 
several coordinated components — for example, 
newspaper advertisements, editorials, and letters to the 
editor; posters; electronic mail messages from student 
health services; contests and other promotional events; 
and group meetings and presentations. Note that a more 
fully developed logic model might also specify the 
frequency and duration of these and other related 
activities. 

The campaign, if successful, should result in several 
immediate (or “proximal”) outcomes. Specifically, 
students will be able to identify the campaigns main 
message and will report that the message is credible. In 
addition, students will report more accurate estimates of 
peer drinking and will cite less social pressure to drink 
heavily. Long-term (or “distal”) outcomes will include 
both a lower prevalence of heavy drinking and fewer 
negative consequences of alcohol consumption (e.g., 
academic failure or unintentional injury). 



An additional outcome might be higher levels of 
support for other prevention initiatives, such as parental 
notification or tougher policies to reduce alcohol 
availability. Here the hypothesis would be that such 
initiatives will gather support as students better 
understand that only a minority of students will be 
impacted rather than a majority. 

A good evaluation will assess whether each of the 
intermediate effects was achieved. In this example, a 
student survey could determine whether, prior to the 
campaign, students actually misperceived drinking 
norms on their campus. Absent that initial 
misperception, a social norms campaign should have 
little effect. Subsequent surveys could assess whether 
students remember the campaign message and find it to 
be credible. If not, then we would not expect the 
campaign to lead to a change in student beliefs about 
peer drinking norms. Survey questions would also be 
included about the other intermediate steps and the 
immediate as well as long-term outcomes. 

It is important to see that the usefulness of the 
evaluation is in large part dependent on its following the 
logic model. If the evaluation were to only measure the 
final outcome, and the intervention fell short of its aims, 
the evaluation would be unable to answer the 



CMC' 



15 



fundamental question of whether the program effects 
were smaller than hoped because the fundamental 
concept behind the intervention was wrong, the 
implementation was flawed, or one piece of the 
intervention sequence fell apart. From a program 
manager's viewpoint, these are crucial distinctions, as the 
answers will suggest different directions to take in the 
future to improve the intervention. 

4. CREATE AND EXECUTE 
A DATA COLLECTION PLAN 

Self-report surveys are a primary data source for program 
and policy evaluations, especially if the goal is to reduce 
consumption or alcohol-related problem behaviors. That 
said, too many program planners assume that evaluation 
data and student survey data are one and the same. As a 
result, they may lose interest in evaluation when there is 
insufficient funding to mount a survey. Even worse, they 
may conduct a poorly administered survey in the hope 
that it will still provide useful data, rather than 
concentrate their resources on developing other potential 
data sources. 

A student drinking survey must meet several 
requirements to be considered scientifically valid. The 
questions themselves must be both valid and reliable, 
meaning that they must truly measure what is intended 
(validity), and that, with a repeated administration, 
students will provide the same answer to the same 
question (reliability) . Fortunately, there are several 
alternative survey instruments available that can be used 
as sources of questions. 

In addition, the survey should be administered at a 
time that reflects typical drinking patterns. Surveys 
conducted shortly after the start of the school year, after 
traditional social events (e.g., homecoming), or close to 



mid-terms or final exams will not provide representative 
data. Hence, most national student drinking surveys are 
conducted in the early part of the spring semester before 
spring break. 

Most important, the sample of students asked to 
participate in the survey must be randomly selected. It 
may be tempting to administer the survey in classrooms, 
but this will not result in a sample that is truly 
representative of all students. There must also be a set of 
procedures in place to boost the response rate. Achieving 
a response rate of 70 percent or more for student surveys 
is extremely difficult. More typical are rates between 50 
and 60 percent. 

It should be remembered, however, that a student 
survey is not the only source of useful data, and in some 
cases may not even be the best source. A key limitation 
is that many of the serious negative consequences of 
drinking are not frequent enough to be well estimated 
by a typical self-report survey involving a sample of only 
200 to 1,000 students. Even so, a sizable university will 
experience these adverse events with some regularity. 

Ideally, colleges and universities will institute a system 
for recording a wide range of alcohol-related incidents 
involving students. For example, it would be of great 
value to have a record of each instance in which a 
student is brought in for urgent or emergency care, 
including an indication of whether alcohol (or other 
drugs) were involved. How this might be done best will 
vary from school to school, depending on the type of 
student health center that is available, the number of 
nearby hospitals, or the manner in which insurance 
claims are processed. 

Similarly, incident reporting forms used by the campus 
police should require officers to indicate whether a 
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student being investigated, cited, or detained has been 
using alcohol. A direct reading of blood alcohol content 
(BAC) using a “passive” breathalyzer, which analyzes 
exhaled air in front of the mouth, would be the best 
means of assessment. Absent that, the officers can be 
asked to make a judgment about alcohol involvement. 
Such judgments can be difficult to make, yet despite 
their fallibility, having such data available over a long 
period of time will still reveal relative changes in alcohol 
involvement that might be attributable to new programs 
and policies. 

Additional examples of potentially useful records are 
listed below. Which types of records are monitored will 
depend on the specific goals and objectives being pursued. 

• Student health services: Costs associated with 
provided medical care when alcohol is involved. 

• Student counseling services: Patient history of 
alcohol use. 

• Residence facilities : Alcohol involvement in 
noise/nuisance complaints, property damage, calls 
for police or emergency services. 

• Athletic department : Alcohol involvement in 
spectator injuries, nuisance complaints, or 
disciplinary actions. 

• Greek student office: Records of alcohol 
involvement in student injuries, noise complaints, 
contacts with police or fire departments, and 
property damage. 

• University discipline: Alcohol involvement in 
charges heard in disciplinary proceedings. 



On many campuses, the problem is that data are 
recorded but are not easily accessed. In some cases, 
program planners may not even be aware of useful 
records being kept by several sub-units of the college or 
university. In other cases, there may only be hard-copy 
records made, which makes compiling the data for an 
evaluation too time-consuming to be practicable. On 
many campuses this situation is improving as offices 
move toward using computerized databases and 
automated data entry. As these systems are put in place, 
administrators should ensure that records of campus 
problems make note of alcohol involvement. 

5. COMMUNICATING 
EVALUATION RESULTS: FEEDBACK 

Our emphasis here has been on the value of evaluation 
for program planning and management. Thus, we again 
stress that the full value of any evaluation is not likely to 
be realized if the information it comprises is not used to 
inform the campus community of what is happening 
with the intervention. In some cases, evidence of the 
programs impact may be mandatory for it to continue. 
There is also the possibility that initial program impact 
may fade with time or that program fidelity will not be 
maintained. 

All communities are deeply interested in whether the 
intervention is effective, but often there are more specific 
questions, too, regarding the effectiveness of each 
component, who is being affected and in what way, and 
how much effort is expended to achieve the impact. 
Support for the intervention often hinges on gathering 
and reporting this information. More broadly, feedback 
tends to engender support even when results are mixed 
or disappointing at first, because providing the 
information builds confidence that the people running 
the program or policy are interested in following 
through to either improve the intervention, or transfer 
resources to an alternative strategy for good reasons. 
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Program Evaluation: The Big Picture 



Newcomers to the topic of college student drinking are 
often puzzled to learn that the field s knowledge of “what 
works” is relatively slim. Apart from some recent and 
promising interventions aimed at individual students, 
the conscientious program planner will find little 
empirical evidence to guide choices of program and 
policy interventions aimed at the broader college 
population. The irony is that this failing is observed 
precisely in those settings — institutions of higher 
education — where the commitment to empirical 
research is high, and expertise in evaluation is readily 
available. 

The broader field of prevention research, which has 
examined the impact of programs and policies aimed at 
youth in the general population, provides useful 
guidance. Indeed, the NIAAA Task Forces report, A Call 
to Action : Changing the Culture of Drinking at U.S. Colleges, 
relied to a great extent on this broader literature when 
identifying recommended prevention strategies for 
higher education administrators. Even so, it is clear that 
evaluations of environmentally focused prevention 
strategies that focus specifically on college populations 
are sorely needed. 

We are urging higher education administrators to 
incorporate evaluation as an integral part of program 
planning, which we view to be essential to developing 
more effective prevention programs and policies. Beyond 
that, however, we hope that administrators will realize 
that the evaluations they undertake will also contribute 
significantly to our knowledge of what works, thereby 
helping other institutions make wise choices when 
designing their programs. There is a common goal that 
all colleges and universities share: to create the 
conditions that will allow students at our Nations 



institutions of higher education to develop their full 
potential. Conducting and then sharing the results of 
evaluations of alcohol prevention efforts is necessary to 
meet that goal. 
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Resources 



The following materials are available from the National 
Institute on Alcohol Abuse and Alcoholism (NIAAA) by 
mail or through the NIAAA Web site 
( www. collegedrinkingprevention.gov ) : 

Task Force Report 

■ A Call to Action: Changing the Culture of Drinking at 
U.S. Colleges 

Final Report of the Task Force on College Drinking 

Panel Reports 

■ High-Risk Drinking in College: What We Know and 
What We Need To Learn 

Final Report of the Task Force on College Drinkings 
Panel on Contexts and Consequences 

■ How To Reduce High-Risk College Drinking: Use Proven 
Strategies, Fill Research Gaps 

Final Report of the Task Force on College Drinkings 
Panel on Prevention and Treatment 

Brochures 

■ What Parents Need to Know About College Drinking 

■ What Peer Educators and Resident Advisors (RAs) Need 
to Know About College Drinking 

■ What Presidents Need to Know About College Drinking 

Future Brochures 

■ What Community Leaders Need to Know About College 
Drinking 



■ What High School Guidance Counselors Need to Know 
About College Drinking 

■ What Students Need to Know About College Drinking 

Planning and Evaluation Handbook 

Reducing Alcohol Problems on Campus: A Guide to 
Planning and Evaluation 

U.S. Department of Educations Higher 
Education Center for Alcohol and Other 
Drug Prevention 

The Higher Education Center is the Nations primary 
resource to assist colleges and universities as they 
develop, implement, and evaluate programs and policies 
to address alcohol and other drug problems on campus. 
The Center provides assistance and information to help 
prevention program planners assess campus needs, 
develop a strategic plan, and measure results. 

Resources available at the Center s Web site 
(http://www.edc.org/hec/eval) include the following: 

Evaluation Links and Resources : This section provides a 
comprehensive list of evaluation resources on a wide 
range of evaluation topics. It includes Center 
publications on evaluation, online evaluation guides, 
comprehensive evaluation Web sites, needs assessment, 
evaluation planning and design, and data collection. 

Resources for Selecting and Working with a Program Evaluator. 
Listed here are resources for prevention professionals to 
assist in choosing and working with an outside evaluator. 

Higher Education Centers Evaluator Database. The referral 
database allows users to search for an evaluator or 
register themselves as an evaluator. 
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Environmental Measurement in Alcohol and Other (AOD) 

Drug Prevention: This section provides presentations and 
other resources on methods for measuring 
environmental change efforts. 

The following publications can be ordered or 
downloaded for free from the Higher Education Center s 
Web site (http://www.edc.org/hec/pubs): 

Understanding Evaluation: The Way to Better Prevention 
Programs (HEC 905). (Note: Only print copies are 
available; they can be ordered through the publications 
section of the Higher Education Centers Web site.) This 
handbook describes the “how and why” of program 
evaluation and outlines the steps involved, working from 
the premise that many useful evaluations can be 
conducted by program staff who may not have formal 
training in evaluation. 

A College Case Study : A Supplement to Understanding 
Evaluation (HEC 904). In telling the story of a fictitious 
college, this case study helps prevention specialists, 
administrators, and others concerned with preventing 
AOD use on college campuses get a feel for what is 
involved in setting up an evaluation of a college AOD 
prevention program and what can be gained from the 
process. 

College Alcohol Risk Assessment Guide : Environmental 
Approaches to Prevention (HEC 109). This guide is 
designed to help college administrators identify factors 
within the campus environment that contribute to 
alcohol-related problems. These factors are examined 
within the context of the public health approach, which 
emphasizes how the environment shapes behavior. 
Methods for identifying problems include scanning. 



analysis, response, and assessment. The publication also 
contains scanning and analysis exercises and selected 
resources. 

Methods for Assessing Student Use of Alcohol and Other 
Drugs (HEC 104). This guide offers a straightforward 
method for gathering and interpreting student survey 
data on alcohol-related problems based on the 
methodology used in a national college alcohol study 
conducted in 1993 by the Harvard School of Public 
Health. It can easily be adapted for all college and 
university campuses. 

Selecting the Right Tool : A Compendium of Alcohol and 
Other Drug Assessment and Evaluation Instruments for Use 
in Higher Education (HEC 114). This compendium 
covers the important issues to consider when selecting 
data collection instruments and describes the leading 
instruments used in the postsecondary AOD prevention 
field. (Sample instruments are included in the printed 
document, but are not available on the Web site.) 

How to Select a Program Evaluator (HEC 716). 

This 3-page flyer describes the role of evaluation in 
program planning and implementation; skills, expertise, 
qualifications, and experience to look for when seeking 
an evaluator; incentives for the evaluator; questions to 
ask when considering an evaluator; and how to network 
to find the right evaluator. 

Online Evaluation Guides 

These guides provide information, advice, and step-by- 
step instructions for conducting program evaluations. All 
can be accessed on the Web. 
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Basic Guide to Program Evaluation 
(http :/ / www. mapnp. org/library/ e valuat n/ fnl_e val . htm) . 
This document provides guidance for planning and 
implementing an evaluation process. Specific topics 
include getting information to make decisions about 
programs; basic ingredients of planning program 
evaluation; evaluating program processes, goals, and 
outcomes; selecting methods; analyzing and interpreting 
information; reporting evaluation results; and pitfalls 
to avoid. 

Community How-To Guide on Evaluation 
(http://nhtsa.dot.gov/people/injury/alcohol/Community). 
This easy-to-use guide, created by the U.S. Department 
of Transportation, includes information on different 
types of evaluation, methods, planning an evaluation, 
and hiring an evaluator. 

Demonstrating Your Programs Worth: A Primer on 
Evaluation (http:// www. cdc . gov/ ncipc/ pu b - 
res/demonstr.htm). This manual, created at the National 
Center for Injury Prevention Control at the Centers for 
Disease Control and Prevention, describes how to 
conduct a simple evaluation, how to hire an outside 
evaluator, and how to incorporate evaluation activities 
into a prevention program. 

Taking Stock : A Practical Guide to Evaluating Your Own 
Programs (http://www.horizon-research.com). This 
manual was created by Horizon Research to help 
community-based organizations design and carry out 
program evaluation. Topics include formative and 
summative evaluation, quantitative and qualitative data, 
and tips for interpreting and reporting data. 



Comprehensive Evaluation Web Sites with Multiple 
Resource Links 

The Web sites listed below contain multiple Web links 
and tools for program evaluation. Additional evaluation 
Web sites can be found at the Higher Education Center’s 
Web site (http://www.edc.org/hec/eval/links.html). 

American Evaluation Association (http://www.eval.org). 
The American Evaluation Association is an international 
professional association of evaluators devoted to the 
application and exploration of program evaluation, 
personnel evaluation, technology evaluation, and many 
other forms of evaluation. Web site resources include 
publications, Web links, reports, surveys, topical interest 
groups, and lists of electronic mailing lists related to 
evaluation, meetings, events, and training. This may 
also be a source for identifying and recruiting evaluation 
specialists. 

Center for Substance Abuse Prevention (CSAP) Decision 
Support System/PreventionDSS 

(http://www.preventiondss.org). PreventionDSS is an 
online training and technical assistance resource 
designed to guide substance abuse prevention 
practitioners through a 7-step planning and evaluation 
process. While the focus is youth substance abuse in a 
community context, the site contains extensive tools and 
resources that are transferable to higher education 
settings. Modules include assessing needs, developing 
strategic plans, building a logic model, choosing 
promising practices, and conducting evaluation. 

Community Toolbox (http://ctb.lsi.ukans.edu). Developed 
at the University of Kansas, this Web site features “how- 
to tools’’ that explain the steps necessary to create and 
evaluate prevention programs. The site also includes 
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sections on strategic planning, community assessment, 
advocacy, grant writing, and evaluation. Each section 
includes a description of the task, advantages of doing it, 
step-by-step guidelines, examples, checklists of points to 
review, and training materials. 

The Evaluation Center (http://www.wmich.edu/evalctr). 
The Evaluation Center, located at Western Michigan 
University, is a research and development unit that 
provides national and international leadership for 
advancing the theory and practice of evaluation. The site 
includes evaluation support services in the form of 
publications, resource links, project descriptions, a 
searchable directory of evaluators, evaluation checklists, 
and a glossary of evaluation terminology. 

Research Methods Knowledge Base 
(http://trochim.cornell.edu/kb/index.htm). This is a 
comprehensive Web-based textbook that addresses all of 
the topics in a typical introductory undergraduate or 
graduate course in social research methods. It covers the 
entire research process including: formulating research 
questions, sampling, measurement, research design, data 
analysis, and report writing. 
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